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Abstract 

In  an  uncertain  world,  a  rational  planning  agent  must  simul¬ 
taneously  reason  with  uncertainty  about  expected  outcomes 
of  actions  and  preferences  for  those  outcomes.  This  work 
focuses  on  systematically  exploring  the  interactions  between 
preferences  for  the  durations  of  events,  and  uncertainty,  ex¬ 
pressed  as  probability  distributions  about  when  certain  events 
will  occur.  We  expand  previous  work  by  introducing  a  means 
for  representing  events  and  durations  that  are  not  under  the 
control  of  the  planner,  as  well  as  quantitative  beliefs  about 
when  those  events  are  likely  to  occur.  Two  reasoning  prob¬ 
lems  are  introduced  and  methods  for  solving  them  proposed. 
First,  given  a  desired  overall  preference  level,  compute  the 
likelihood  that  a  plan  exists  that  meets  or  exceeds  the  spec¬ 
ified  degree  of  preference.  Second,  given  an  initial  set  of 
beliefs  about  durations  of  events,  and  preferences  for  times, 
infer  a  revised  set  of  preferences  that  reflect  those  beliefs. 

Introduction 

Rational  agents  are  capable  of  mentally  exploring  the  in¬ 
teractions  between  what  they  believe  and  what  they  desire 
as  outcomes  of  actions.  More  often  than  not,  the  value  of 
the  outcomes  of  actions  cannot  be  described  by  a  single  at¬ 
tribute,  but  rather  by  attributes  that  combine  to  determine  the 
overall  value  of  the  outcome  (Keeney  &  Raifa  1993).  Fur¬ 
thermore,  the  outcome  of  actions  may  not  be  known  with 
certainty,  as  a  result  of  the  need  to  interact  with  the  world. 

Many  practical  planning  or  scheduling  problems  surround 
events  that  are  not  controlled  by  the  planning  agent.  For  ex¬ 
ample,  Earth  Science  observation  scheduling  may  involve 
assigning  times  for  the  remote  sensing  of  an  area  of  inter¬ 
est  on  the  Earth  before,  during,  or  after  a  fire  has  occurred 
there.  The  start  and  end  of  the  fire  are  not  known  with  cer¬ 
tainty  at  planning  time,  but  Earth  Science  models  might  be 
available  to  estimate  a  set  of  times  when  fires  are  likely  to 
occur.  In  addition,  the  scientific  utility  of  an  observation 
may  vary  based  on  when  the  observation  is  taken  relative  to 
the  fire,  resulting  in  preferences  for  temporal  orderings  and 
durations  between  planned  events  and  uncontrollable  events 
(Morris  et  al.  2004b).  As  automated  planning  matures  as 
a  software  technology,  new  techniques  inspired  by  decision 
theory  are  being  integrated  to  address  the  fact  that  plans  are 
executed  in  the  world,  with  varying  degrees  of  value  to  the 
planner  based  on  their  outcomes  (Blythe  1999).  A  princi¬ 


pled  approach  to  scheduling  problems  as  the  above  is  essen¬ 
tial  for  a  decision-theoretic  temporal  planner  that  takes  into 
account  preferences  when  determining  plan  quality. 

The  goal  in  this  paper  is  to  devise  systematic  methods  for 
exploring  the  interactions  between  temporal  preferences  and 
uncertainties.  We  introduce  a  framework  that  generalizes 
the  Simple  Temporal  Problem  (STP)  formulation  (Dechter, 
Meiri,  &  Pearl  1991),  called  the  Simple  Temporal  Problem 
with  Preferences  and  Probabilities ,  or  STP3.  One  com¬ 
ponent  of  the  generalization  adds  the  capability  to  express 
preferences  for  times,  following  (Khatib  et  al.  2001).  The 
other  component  allows  for  the  designation  of  uncontrol¬ 
lable  events  and  the  associated  probability  space  over  times. 
We  extend  techniques  previously  used  to  solve  temporal 
planning  problems  with  preferences  to  identify  solutions  that 
are  both  globally  preferred  and  highly  probable. 

Besides  defining  the  STP3  framework,  the  contribution  of 
this  paper  is  to  describe  solutions  to  two  practical  reasoning 
problems  arising  from  the  interactions  between  probabilities 
and  preferences.  We  extend  techniques  previously  used  to 
solve  temporal  problems  with  preferences  to  identify  solu¬ 
tions  that  are  both  globally  preferred  and  highly  probable. 

Decision-theoretic  planning  is  surveyed  by  (Blythe  1999). 
Most  approaches  extend  classical  planning  techniques  or 
employ  Markov  Decision  Processes  (e.g.  (Boutilier,  Dean, 
&  Hanks  1999)),  in  contrast  to  our  constraint-based  focus. 
Of  work  on  temporal  reasoning  for  planning,  a  characteris¬ 
tic  example  is  (Hanks,  Madigan,  &  Gavrin  1995),  who,  like 
us,  consider  exogenous  events,  but  focus  on  eliciting  proba¬ 
bilities  and  qualitative  preferences  from  a  human  expert. 

In  the  constraints  literature,  preferences  are  commonly 
represented  using  semiring-based  formulations,  the  ap¬ 
proach  we  adopt.  An  alternative  formulation  for  qualitative 
preferences  is  CP-nets  (Boutilier  et  al.  2004).  Uncertainty 
has  also  been  represented  both  qualitatively  and  quantita¬ 
tively;  probabilistic  frameworks  include  that  of  (Fargier  et 
al.  1995),  which  we  adopt,  and  its  extensions. 

Generic  constraint-based  frameworks  that  account  for 
both  preferences  and  uncertainty  include  (Dubois,  Fargier, 
&  Prade  1996).  Our  work  is  distinguished  by  restricting 
attention  to  Simple  Temporal  constraints.  Prior  work  in 
this  line  has  considered  STPs  with  preferences  but  no  un¬ 
certainty  (Khatib  et  al.  2001);  and  STPs  with  uncertainty 
constraints  but  no  preferences  (Morris,  Muscettola,  &  Vidal 
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2001;  Tsamardinos,  Pollack,  &  Ramakrishnan  2003).  While 
(Rossi,  Venable,  &  Yorke-Smith  2004)  incorporate  both  as¬ 
pects,  that  work  considers  only  qualitative  uncertainty,  that 
is,  with  implied  uniform  distributions. 

Example:  Earth  Science  Campaign 
Observation  Scheduling 

An  Earth  Science  campaign  is  a  systematic  set  of  activities 
undertaken  to  meet  a  particular  science  objective.  Here,  we 
present  a  hypothetical  campaign  based  on  a  science  objec¬ 
tive  to  test  an  emissions  model  predicting  the  aerosols  re¬ 
leased  by  wildfires.  Data  on  several  variables  must  be  gath¬ 
ered  in  order  to  accomplish  the  analysis,  and  several  remote 
sensors,  such  as  those  on  the  Landsat  satellite,  provide  data 
products  at  various  spatial  resolutions  relevant  to  these  vari¬ 
ables.  Preferred  times  for  acquiring  Landsat  data  for  vegeta¬ 
tion  type  for  a  region  of  interest  in  the  northern  hemisphere 
would  be  the  prior  June  or  July  in  the  same  year  that  the 
fire  burned,  when  forested  land  can  most  easily  be  spectrally 
distinguished  from  grassland.  For  mapping  aerosol  concen¬ 
tration,  images  coincident  to  burning  must  be  obtained;  the 
Terra  and/or  Aqua  satellites  have  relevant  instruments.  For 
the  burned  area,  data  should  be  acquired  after  (though  not 
too  long  after)  the  fire  is  out,  while  for  mapping  vegetation 
moisture  content,  hyperspectral  data  from  an  EO-1  Hyper¬ 
ion  instrument  are  relevant,  and  the  most  useful  data  would 
be  that  acquired  just  preceding  the  fire. 

From  this  description,  the  inputs  to  a  campaign  planning 
problem  potentially  consist  of  the  following  characteristics: 

•  a  set  of  temporal,  spatial,  and  resource  constraints  on 
when  and  where  images  are  to  be  taken; 

•  user  preferences  for  when  an  observation  should  be  taken; 

•  temporal  ordering  constraints  between  planned  events  and 
uncontrollable,  exogenous  events  such  as  fires. 

A  reasonable  goal,  given  these  inputs,  is  to  generate  a  con¬ 
cise  representation  of  the  set  of  solutions  (assignments  of 
times  and  sensing  resources)  that  are  maximally  preferred 
and  reflect  a  set  of  initial  beliefs  about  when  exogenous 
events  are  likely  to  occur.  We  formulate  a  framework  ca¬ 
pable  of  describing  the  problem  and  generating  this  output. 

Simple  Temporal  Problems  with  Preferences 
and  Probabilities 

A  soft  temporal  constraint  depicts  restrictions  on  the  dis¬ 
tance  between  arbitrary  pairs  of  distinct  events,  and  a  user- 
specified  preference  for  a  subset  of  those  distances.  In 
Khatib  et  al.  (Khatib  et  al.  2001),  a  soft  temporal  constraint 
between  events  i  and  j  is  defined  as  a  pair  (I,  fj),  where  I  is 
a  set  of  intervals  { [a,  b] ,  a  <  6}  and  f.t]  is  a  local  preference 
function  from  /  to  a  set  A  of  admissible  preference  values.1 
When  I  is  a  single  interval,  a  set  of  soft  constraints  defines  a 
Simple  Temporal  Problem  with  Preferences  (STPP),  a  gen¬ 
eralization  of  a  Simple  Temporal  Problem  (Dechter,  Meiri, 

'For  the  purposes  of  this  paper,  we  assume  the  values  in  A  are 
totally  ordered,  and  that  A  contains  designated  values  for  minimum 
and  maximum  preference. 


Figure  1 :  STP3  Representing  the  Fire  Campaign  Scenario 


&  Pearl  1991).  An  STPP  can  be  depicted  as  a  pair  (V,  C) 
where  V  is  a  set  of  variables  representing  events  or  other 
time -points,  and  C  =  { ( \at] ,  bt.j] ,  fj ) }  is  a  set  of  soft  con¬ 
straints  defined  over  V.  An  STPP,  like  an  STP,  can  be  or¬ 
ganized  as  a  network  of  variables  representing  events,  and 
links  labeled  with  constraint  information. 

Similar  to  other  recent  approaches  (Morris,  Muscettola,  & 
Vidal  2001;  Tsamardinos,  Pollack,  &  Ramakrishnan  2003; 
Rossi,  Venable,  &  Yorke-Smith  2004),  we  extend  the  STPP 
framework  to  represent  temporal  uncertainty.  First,  we  parti¬ 
tion  V  into  two  groups:  the  decision  variables  V,i  and  the  pa¬ 
rameters  Vu  representing  uncontrollable  events.  This  parti¬ 
tion  induces  a  distinction  between  decision  constraints  (C,i) 
and  uncertainty  constraints  ( Cu ):  those  constraints  whose 
end-point  is  controllable  (i.e.  a  decision  variable),  and  those 
whose  end-point  is  a  uncontrollable  (i.e.  a  parameter).  An 
uncertainty  constraint  depicts  a  duration  between  events  as  a 
continuous  random  variable.  To  ease  the  exposition,  we  as¬ 
sume  that  the  uncertainty  constraints  are  mutually  indepen¬ 
dent2;  this  allows  the  constraints  in  Cu  to  be  expressed  in 
the  form  ([a^-,  6^],^),  where ptj  :  [a.y,  bif  ->  [0, 1]  is  the 
probability  density  function  over  the  designated  interval.  We 
call  the  framework  (Vd,  Vu,  Cd,  Cu ),  where  Cd  are  soft  con¬ 
straints,  a  Simple  Temporal  Problem  with  Preferences  and 
Probabilities,  or  STP3. 

Example  1  Earth  Science  Observation  Problem.  Inputs: 
Variables  in  V,/  standing  for  two  controllable  events  consist¬ 
ing  of  taking  an  observation  (Obs  1,  Obs2),  and  two  uncon¬ 
trollable  events  in  Vu,  the  start  and  end  of  a  fire  ( FS,  FE ) 
(for  simplicity,  observations  are  viewed  as  instantaneous),  as 
shown  in  Figure  1 .  There  is  also  an  event  TR  representing 
the  beginning  of  time.  Soft  constraints  /i(t),  /2(f)  in  Cd.  are 
associated  with  the  durations  between  Obs  1  and  FS,  and 
between  Obs2  and  FE,  respectively.  For  example,  /)  (t) 
may  express  that  there  is  no  value  for  taking  Obs  1  after  the 
start  of  the  fire  (FS),  and  a  preference  for  times  that  are  as 
close  to  FS  as  possible.  Similarly,  /2(f)  expresses  a  pref¬ 
erence  for  Obs2  happening  before  FE  as  close  as  possible, 

2For  instance,  imagine  that  the  Earth  Science  planner  maintains 
a  Bayes  network  elsewhere  to  express  the  dependencies;  each  prob¬ 
ability  p(t)  is  given  implicitly  by  that  network. 


with  a  penalty  if  the  observation  is  taken  after  the  fire.  Un¬ 
certainty  constraints  pi,  p2  in  Cu  are  associated  with  random 
variables  representing  the  start  time  and  the  duration  of  the 
fire.  These  constraints  are  based  on  Earth  Science  models 
about  fires  in  the  area  of  interest.  For  example,  p\  may  ex¬ 
press  a  normal  distribution  over  the  range  of  times. 

A  solution  to  an  STP3  is  a  set  of  assignments  to  V  = 
Vfi  U  Vu  that  satisfies  all  the  constraints  in  C  =  Cd  U  Cu. 
Given  an  STP3  P,  let  Sol(P)  be  the  set  of  all  solutions  to  P. 
An  arbitrary  solution  s  £  Sol(P)  can  be  viewed  as  having 
two  parts:  Sd,  the  set  of  values  assigned  to  V,i,  and  su,  the 
set  of  values  assigned  to  Vu . 

Our  goal  is  to  develop  efficient  methods  for  generating 
a  concise,  graphical  representation  of  subsets  of  Sol(P) 
corresponding  to  highly  likely,  globally  preferred  solutions. 
This  STP-based  graphical  representation  is  called  a  flexible 
(temporal)  plan.  Many  planning  systems  use  an  STP-based 
representation  of  the  temporal  aspects  of  their  plans  (Smith, 
Frank,  &  Jonsson  2000). 

Following  previous  efforts,  methods  for  flexible  tempo¬ 
ral  planning  under  uncertainty  can  be  distinguished  based 
on  assumptions  about  the  strategy  to  be  applied  in  executing 
the  flexible  plan.  A  static  execution  strategy  assumes  no  ac¬ 
cess  to  the  values  of  su  during  plan  execution;  by  contrast  a 
dynamic  execution  strategy  is  applied  as  plan  execution  pro¬ 
ceeds  and  the  values  of  su  are  observed  over  time  (Morris, 
Muscettola,  &  Vidal  2001;  Rossi,  Venable,  &  Yorke-Smith 
2004).  The  results  of  this  paper  assume  a  static  execution 
strategy;  we  defer  discussions  of  planning  for  dynamic  exe¬ 
cution  of  STP3s  to  future  work. 

Component  Solvers.  The  solution  methods  described  be¬ 
low  are  based  on  different  decompositions  of  an  STP3  into 
component  sub-problems  for  which  efficient  solution  meth¬ 
ods  exist.  As  a  final  preliminary,  we  fix  some  terminology 
and  briefly  summarize  these  sub-problem  solution  methods. 
Given  an  STP3,  the  underlying  STPP  is  the  problem  that  re¬ 
sults  when  a  constraint  {[a,b\,pxy}  £  Cu  is  replaced  by 
the  STP  component  constraint  [a,b\.  The  underlying  Prob¬ 
abilistic  STP  is  the  problem  that  results  when  each  soft  con¬ 
straint  {[a,  b],  fxy}  £  Cd  is  replaced  by  the  STP  compo¬ 
nent  constraint  [a,  b\.  The  underlying  STP  replaces  all  con¬ 
straints  in  Cd  U  Cu  with  their  STP  components. 

Efficient  solution  methods  for  STPs  are  well-known 
(Dechter,  Meiri,  &  Pearl  1991).  An  Simple  Temporal  Net¬ 
work  (STN)  is  a  graph  of  nodes  representing  the  STP  vari¬ 
ables  and  edges  labeled  with  the  interval  temporal  con¬ 
straints.  Each  STN  is  associated  with  a  distance  graph  de¬ 
rived  from  the  upper  and  lower  bounds  of  the  interval  con¬ 
straints.  An  STN  is  consistent  iff  the  distance  graph  does  not 
contain  a  negative  cycle;  this  condition  can  be  determined 
by  applying  a  single-source  shortest  path  algorithm  such  as 
Bellman-Ford.  In  addition  to  consistency,  it  is  often  useful 
to  determine  for  an  STN  the  equivalent  STN  (in  terms  of  a 
set  of  solutions)  in  which  all  the  intervals  are  as  “tight”  as 
possible.  This  minimal  network  can  be  determined  by  ap¬ 
plying  an  All-Pairs  Shortest  Path  (APSP)  algorithm  to  the 
input  network  (Dechter,  Meiri,  &  Pearl  1991). 


Figure  2:  Illustrating  the  Interactions  between  Temporal 
Probabilities  and  Preferences 


Previous  efforts  in  solving  STPPs  have  been  based  on 
identifying  and  applying  criteria  for  “globally  preferred  so¬ 
lutions”  such  as  “weakest  link”  (maximize  the  least  pre¬ 
ferred  local  preference),  “pareto”,  and  “utilitarian”  (Mor¬ 
ris  et  al.  2004a).  Developing  efficient  solvers  has  required 
local  preference  functions  that  are  linear  or  semi-convex.3 
One  method  for  solving  STPPs  efficiently  is  called  the  chop 
method ,  first  introduced  in  (Khatib  et  al.  2001).  The  chop 
method  is  a  two-step  search  process  of  iteratively  choosing 
a  preference  value  a ,  “chopping”  every  preference  function 
at  that  point,  and  then  solving  an  underlying  STP  defined  by 
the  interval  of  temporal  values  whose  preference  values  lie 
above  the  chop  line,  i.e.  {x  :  f(x)  >  a};  henceforth,  we 
refer  to  this  as  the  chop  interval.  The  highest  chop  point  that 
results  in  a  solvable  (i.e.  consistent)  STP  produces  a  flexible 
plan  whose  solutions  are  exactly  the  optimal  solutions  of  the 
original  STPP  (based  on  the  criteria  of  weakest  link).  Binary 
search  can  be  used  to  select  candidate  chop  points,  making 
the  technique  for  solving  the  STPP  tractable. 

Assessing  the  Likelihood  of  Achieving 
Preferred  Plans 

This  section  and  the  next  consider  two  practical  reasoning 
problems  involving  the  interactions  of  uncertainty  and  pref¬ 
erences  about  time,  and  demonstrate  how  under  certain  as¬ 
sumptions  they  can  be  solved  efficiently  using  STP3s.  The 
first  problem  addresses  the  question:  what  are  the  chances 
of  achieving  a  certain  level  of  global  preference,  given  my 
belief  about  the  way  the  world  will  behave?  To  illustrate, 
consider  the  simple  STP3  in  Figure  2(a).  Here,  Vd  =  {A,B} 
and  Vu  =  { C } ,  and  there  are  two  decision  constraints,  be¬ 
tween  B  and  C  and  between  A  and  B.  B  is  tightly  con¬ 
strained  to  occur  exactly  one  time  unit  after  A.  The  soft 
constraint  BC  prefers  durations  between  B  and  C  to  be 
minimal  (higher  values  more  preferred);  this  is  expressed  by 
the  preference  function  /(f)  =  4  —  t.  The  probability  den¬ 
sity  function  for  AC  is  represented  by  specifying  the  named 
function  (normal)  with  mean  (3)  and  standard  deviation  (1). 

3  A  function  is  semi-convex  if  drawing  a  horizontal  line  any¬ 
where  in  the  Cartesian  plane  of  the  graph  of  the  function  is  such 
that  the  set  of  X  such  that  f(X)  is  not  below  the  line  forms  an  in¬ 
terval.  Semi-convexity  ensures  that  there  is  a  single  interval  above 
any  chop  point,  and  hence  that  the  resulting  problem  is  an  STP. 


Figure  3:  Why  the  Upper  Bound  May  Not  be  Tight 

Suppose  an  agent  wants  to  infer  the  chances  of  there  being 
a  solution  with  an  overall  preference  level  of  2  or  greater.  We 
can  answer  this  question  by  restricting  assignments  to  BC 
with  an  /  value  of  2  or  greater,  and  propagating  the  temporal 
constraints  over  the  network.  This  means  shrinking  the  BC 
interval  to  [0,2],  which  in  turn  shrinks  AC  to  [1,3].  Conse¬ 
quently,  the  answer  to  the  posed  question  can  be  obtained  by 
computing  P(1  <  t  <  3)  =  J)3  p{t)dt. 

This  technique  can  be  generalized  for  arbitrary  STP3s. 
Given  an  STP3  P,  to  determine  the  probability  of  achieving 
a  solution  of  global  preference  value  7  or  higher,  we  perform 
the  following  procedure: 

1.  Given  an  input  STP3,  chop  each  local  preference  function 
at  the  designated  preference  value  7.  Form  a  new  problem 
by  replacing  each  associated  interval  with  the  resulting 
chop  interval. 

2.  Determine  the  minimal  network  of  the  underlying  STP  of 
the  new  problem,  using  an  APSP  algorithm. 

3.  Compute  the  overall  probability  of  the  underlying  proba¬ 
bilistic  CSP.  Assuming  independence  of  the  pl3 ,  the  value 
to  be  computed  is 

Y[P(aij  <t<bi:i),  (1) 

Pij 

where  for  each  uncertainty  constraint,  [al3 ,  bij)  is  the  in¬ 
terval  of  the  minimal  network  derived  from  step  2. 

Provided  step  3,  which  may  be  done  using  numerical  in¬ 
tegration,  is  of  polynomial  complexity,  the  whole  method 
is  polynomial.  Steps  2  and  3  of  this  method  resemble  the 
method  proposed  in  (Tsamardinos,  Pollack,  &  Ramakrish- 
nan  2003)  for  solving  Probabilistic  STPs.  Unfortunately,  it 
can  be  easily  shown  that  the  computed  value  provides  only 
an  upper  bound  on  the  probability  that  the  solutions  defined 
at  that  chop  level  or  above  will  succeed.  That  this  is  not  a 
tight  upper  bound  can  be  demonstrated  by  a  simple  example, 
found  in  Figure  3.  In  this  example,  chopping  the  preference 
function  at  10  and  solving  the  underlying  STP  would  not 
shrink  the  temporal  bounds  of  the  uncertainty  links.  There¬ 
fore,  the  probability  of  succeeding  returned  by  this  method 
would  be  1,  although  in  fact  some  of  the  probability  mass  is 
lost  as  a  result  of  the  chop. 


Despite  these  limitations,  an  upper  bound  computation 
may  be  useful;  if  the  bound  is  too  low,  the  planner  will  be 
forced  to  “lower  expectations”  of  the  plan  branch  under  con¬ 
sideration,  i.e.  its  overall  expected  preference  level. 

A  tighter  bound  would  require  examining  the  mass  of  the 
polytope  defined  by  all  the  constraints  (a  similar  observa¬ 
tion  was  made  in  (Tsamardinos,  Pollack,  &  Ramakrishnan 
2003)).  Applied  to  the  previous  example,  we  get  P(( 0  < 
AB  <  10)  A  (0  <  BC  <  10))  from  (1),  but  the  true  proba¬ 
bility  is  P((0  <  AB  <  10)  A  (0  <  BC  <  10)  A  (AB  +  BC  > 
10))  or  simply  P(AB  +  BC  >  10),  assuming  the  bounds. 
(Note  that  the  AB  and  BC  random  variables  are  no  longer 
independent  under  the  condition  AB  +  BC  >  10.)  We  can 
reformulate  this  as  P(\J X(AB  =  x  A  BC  >  10  —  2;))  and 
calculate  it  as 

/  (  /  P(y)dy )  p{x)dx. 

J 0  \J10-X  J 

Inducing  Preferences  from  Probabilities 

In  this  section  we  consider  a  sort  of  dual  problem  to  that 
posed  in  the  previous  section:  given  current  expectations 
about  the  world,  how  can  preferences  be  systematically  ad¬ 
justed  to  fit  with  those  expectations?  Intuitively,  by  answer¬ 
ing  this  question,  the  planner  can  “factor  out”  the  temporal 
uncertainty  in  the  problem,  resulting  in  a  pure  decision  prob¬ 
lem:  because  of  the  factoring,  the  solutions  most  preferred 
based  on  the  induced  preferences  are  also  most  likely. 

This  factoring  process  takes  into  account  the  initial  pref¬ 
erences  on  decision  constraints,  combining  them  with  the 
preferences  induced  from  the  uncertainty  constraints.  The 
core  idea  is  to  apply  the  concept  of  expected  utility  from  de¬ 
cision  analysis  (Keeney  &  Raifa  1993)  to  represent  induced 
local  preferences.  Once  the  reasoning  is  complete,  the  “out¬ 
put”  preferences  on  the  decision  constraints  thus  reflect  both 
the  preferences  of  the  agent  and  its  expectation  about  the 
uncertainty  in  the  world. 

The  main  result  of  this  section  will  be  to  state  a  set  of  suf¬ 
ficient  conditions  for  finding  an  efficient  algorithm  for  this 
process  for  certain  classes  of  STP3.  The  method  consists  of: 

1.  Given  an  input  STP3,  derive  the  minimal  network  of  the 
underlying  STP. 

2.  Apply  a  local  consistency  algorithm  (discussed  below)  to 
the  resulting  STP3  (i.e.  with  the  tightened  interval  con¬ 
straints)  to  compute  the  induced  preferences. 

3.  Solve  the  underlying  STPP  of  the  resulting  network  using 
the  chop  solver  to  find  the  globally  preferred  solutions. 

The  set  of  solutions  making  up  the  flexible  plan  that  results 
are  the  expected  globally  preferred  solutions. 

To  examine  the  second  step  in  more  detail,  we  mimic  the 
method  of  triangular  reduction  found  in  (Morris,  Muscet- 
tola,  &  Vidal  2001),  used  to  solve  Simple  Temporal  Prob¬ 
lems  with  Uncertainty  (STPUs).  We  consider  all  STP3s  as 
collections  of  triangular  subnetworks  of  the  form  illustrated 
by  Figure  2(b),  where  there  is  a  single  uncertainty  constraint 
on  AC  with  bounds  [it,  v],  and  two  decision  constraints  on 
AB  and  BC  with  bounds  [ y ,  2]  and  [w,  x\  respectively.  As 
in  the  Earth  Science  example,  A  might  be  the  beginning  of 


time,  B  might  be  the  start  of  a  planned  observation,  and 
C  the  onset  of  a  fire.  The  goal  is  to  compute  the  regres¬ 
sion  of  pac  over  fnc  to  find  the  induced  soft  decision  con¬ 
straint  Jab-  (The  case  in  which  AB  is  also  associated  with  a 
soft  constraint  can  be  handled  as  part  of  the  general  solution 
method  discussed  later.) 

To  handle  the  single  triangle  case,  we  need  to  consider 
three  possible  orderings  between  B  and  C.  We  assume  that 
step  1  of  the  approach  has  been  applied,  so  that  the  triangular 
network  has  been  minimized.  If  B  precedes  C  (w  >  0),  then 
the  induced  soft  constraint  is  {[y,  z ],  Jab},  where 

fAB{t)=  f  f{t'  -t)p{t')dt'. 

J  U 

Although  in  general  this  function  cannot  be  derived  analyt¬ 
ically,  with  certain  restrictions  placed  on  the  shape  of  the 
preference  function  it  may  be  possible  to  compute  it  directly. 
Alternatively,  we  can  estimate  it  numerically  (e.g.  Monte 
Carlo  integration),  or  even  perform  crude  but  fast  estimation 
based  on  the  expected  value.  If  C  precedes  B  (x  <  0),  then 
intuitively  the  planner  does  not  require  any  knowledge  about 
the  expected  time  of  C  in  order  to  deduce  the  preferred  time 
to  execute  B  dynamically  (the  soft  constraint  on  AB  in  this 
case  can  be  derived  from  that  of  BC ).  However,  recall  that 
we  focus  only  on  the  situation  of  static  execution,  in  which 
knowledge  about  C  is  not  available  at  planning  time.  This 
means  that  the  predictive  models  of  the  Precede  case  are  rel¬ 
evant  to  planning  the  Follow  case:  the  same  technique  can  be 
followed.  Finally,  for  static  execution  the  same  also  applies 
if  B  and  C  are  unordered  (w  <0.  x  >  0). 

To  derive  the  induced  constraints  for  general  STP3  net¬ 
works,  we  consider  all  triangles  separately,  propagating  the 
effects  of  one  operation  to  neighboring  triangles,  until  the 
network  is  quiescent.  Thus,  the  stmcture  of  the  algorithm  is 
similar  to  determining  path-consistency  in  an  STP  network. 
Propagation  requires  combining  local  preference  functions. 
The  same  combination  operator  as  that  used  for  determin¬ 
ing  local  consistency  for  preference  networks  (Rossi  et  al. 
2002)  can  be  applied  here  for  propagating  soft  constraints. 
After  the  network  has  reached  quiescence,  the  planner  can 
safely  discard  the  probability  density  functions  pxy  in  Cu. 
Removing  them  results  in  the  underlying  STPP,  which  can 
be  solved  by  the  chop  method  (Khatib  et  al.  2001). 

The  following  result  summarizes  these  core  ideas.  It  will 
be  proved  informally  and  illustrated  by  an  example.  Follow¬ 
ing  terminology  in  (Morris,  Muscettola,  &  Vidal  2001),  an 
STP3  will  be  said  to  be  pseudo-controllable  if  no  interval  in 
an  uncertainty  constraint  is  “squeezed”  as  the  result  of  per¬ 
forming  step  1  above  (computing  the  minimal  network).  We 
refer  to  the  STP3  that  results  from  performing  step  2  above 
as  the  induced  STP3. 

Theorem  1  Given  an  STP3  with  the  following  properties: 

1.  The  input  preference  functions  p.tj  are  linear  or  semi- 

convex  piecewise  linear  (intuitively,  semi-convex  piece- 

wise  linear  means  that  there  are  no  “V”  shaped  segments); 

2.  The  STP3  is  pseudo-controllable; 

3.  The  probability  distributions  on  the  uncertainty  con¬ 
straints  are  normal; 


Figure  4:  Example  of  Induced  Preferences 

then,  using  the  method  described  above,  the  set  of  expected 
globally  preferred  solutions  to  the  initial  STP3  can  be  com¬ 
puted  in  polynomial  time. 

The  first  condition  of  the  theorem  is  needed  to  ensure  that 
the  induced  STP3  has  only  functions  that  are  semi-convex, 
which  is  required  for  the  application  of  the  chop  solver 
method  in  step  3  (a  polynomial-time  procedure).  Steps  2 
and  3  are  required  to  simplify  the  induced  functions  to  lin¬ 
ear  functions  involving  expected  values  (see  the  example  be¬ 
low).  The  conclusion  of  the  proof  consists  of  observing  that 
the  underlying  procedures  applied  in  the  method  (all-pairs 
shortest  path,  the  local-consistency  technique  for  deriving 
induced  preferences,  the  chop  solver,  and  numerical  integra¬ 
tion  for  determining  the  expected  values)  are  all  polynomial. 

To  illustrate  step  2  of  the  method  in  the  general  case,  con¬ 
sider  the  STP3  in  Figure  4.  This  problem  consists  of  two 
decision  constraints  on  BC  and  BD  with  associated  prefer¬ 
ence  functions  /,  g  defined,  /  clearly  preferring  larger  dura¬ 
tions  between  B  and  C,  and  g  preferring  smaller  durations. 
Two  uncertainty  constraints  on  AC  and  AD  consist  of  nor¬ 
mal  probability  density  functions  pi  and  p2  with  means  and 
standard  deviations  indicated  in  parentheses.  The  goal  is  to 
infer  the  induced  preference  function  h  on  AB  (the  network 
is  already  minimal). 

First,  considering  the  triangle  ABC,  one  induced  function 
for  h  arises  as  follows: 

/■10 

hi(t)  =  /  f{t'  -t)px{t')dt' 

Jo 

rlO 

=  /  [f -t]pl{t')dt' 

Jo 

r 10  rlO 

=  /  t'pi{t')dt'  —  t  /  pi{t')dt' . 

Jo  Jo 

Notice  that  because  of  the  pseudo-controllability  of  the 
network  (it  being  already  minimal),  the  last  equation  re¬ 
duces  to  E(T\)  —  t,  since  then  pi(t')dt'  =  1  and 

f0W  t'pi{t')dt'  =  E(Ti),  where  E(T\)  is  the  expected  value 
of  the  random  variable  1\  associated  with  the  duration.  A 
similar  derivation  based  on  the  triangle  ABD  then  results  in 
another  induced  function  /12(f)  =  10  —  \E{T2)  —  t\.  The 


final  induced  function  h  becomes  the  combination  of  hi  and 
hi',  e.g.  the  intersection  of  the  areas  under  the  functions. 

This  approach  can  be  generalized  for  regression  over 
semi-convex  piecewise  linear  preference  functions.  Let  fnc 
be  the  intersection  of  n  linear  segments  f}}C , . . .  /gc,  where 
for  each  k,  [a%c^Bc\  is  the  segment  for  which  fsc  = 
f%c-  When  regressing  pac  over  /bc  to  compute  the  in¬ 
duced  preference  function  hAB ,  we  have: 

hAB{t)  =  [  fBc(t'  ~  t)pAc(t')dt' 

Ja 1 

b fe 

=  [  feci*'  ~t)pAc(t')dt', 

k=l,...,n.ak 

which  simplifies  to  sums  involving  linear  functions. 

This  example  shows  how  with  suitable  restrictions  on  the 
shapes  of  the  preference  functions  and  on  whether  the  all¬ 
pairs  computation  eliminates  any  of  the  probability  mass,  the 
computation  of  induced  preferences  can  be  made  efficient. 

Discussion  and  Future  Work 

We  have  examined  temporal  reasoning  under  the  interac¬ 
tions  of  preferences  and  quantitative  uncertainty  in  the  con¬ 
text  of  constraint-based  planning.  In  addition  to  the  formu¬ 
lation  of  the  STP3  framework,  which  augments  the  Simple 
Temporal  Problem  with  both  preferences  and  probabilities, 
the  main  contribution  of  this  paper  is  to  formulate  two  plan¬ 
ning  decision  problems.  Utilizing  standard  methods  from 
decision  theory,  probability  theory,  and  recent  advances  in 
constraint  satisfaction,  we  have  shown  how  flexible  tempo¬ 
ral  plans  can  be  generated  that  are  most  preferred  based  on 
what  the  planning  agent  believes  about  the  expected  times  of 
events;  and  how  the  agent  can  update  its  preferences,  given 
its  beliefs. 

Fundamentally,  preferences  and  uncertainty  are  orthogo¬ 
nal  aspects  of  the  decision  problem.  Both  planning  decisions 
we  have  considered  are  approaches  to  combining  the  two 
aspects;  which  is  most  relevant  depends  on  the  aim  of  the 
planning  agent  and  the  questions  being  asked  of  it.  The  first 
decision,  to  evaluate  the  probability  of  a  plan  existing  with 
at  least  a  given  preference,  is  useful  to  determine  whether  a 
plan  branch  can  meet  a  minimum  quality  threshold.  The  sec¬ 
ond  decision,  to  update  preferences  based  on  beliefs,  is  use¬ 
ful  to  factor  the  uncertainty  into  a  single  criterion  for  plan 
evaluation.  Besides  these  two  decision  problems,  the  pro¬ 
posed  framework  can  be  applied  to  related  problems;  for  in¬ 
stance,  an  agent  might  seek  to  determine  the  maximal  prefer¬ 
ence  level  at  which  a  solution  exists  with  a  given  probability 
p.  When  p  =  1  and  the  probabilities  are  uniform,  this  cor¬ 
responds  to  certain  forms  of  strong  controllability  addressed 
in  (Rossi,  Venable,  &  Yorke-Smith  2004). 

Future  theoretical  efforts  include  characterizing  more 
fully  the  computational  complexity  of  STP3s,  and  refining 
the  bound  on  the  probability  that  a  plan  exists  with  given 
preference  quality.  In  addition  to  implementing  the  methods 
described  in  this  paper,  our  major  next  step  is  to  extend  the 
results  here  to  address  issues  in  planning  under  a  dynamic 


execution  strategy.  Of  particular  importance  will  be  to  ex¬ 
amine  the  interactions  between  preferences  and  wait  con¬ 
straints  that  emerge  when  determining  the  controllability  of 
flexible  plans,  as  described  in  (Morris  &  Muscettola  2000). 
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